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Abstract 



We establish an asymptotic formula for determinants of truncated Wiener-Hopf 
+Hankel operators with symbol equal to the exponential of a constant times the char- 
I acteristic function of an interval. This is done by reducing it to the corresponding 

■ (known) asymptotics for truncated Toeplitz+Hankel operators. The determinants in 

, question arise in random matrix theory in determining the limiting distribution for 

I the number of eigenvalues in an interval for a scaled Laguerre ensemble of positive 

' Hermitian matrices. 

o 



I. Introduction 



It has long been known that the asymptotic evaluation of Fredholm determinants of 
truncated Wiener-Hopf operators Wnic) is important in the study of certain random matrix 
' problems. These operators act on L'^(0,R) according to the rule 

rR 

f{x)^g{x) = f{x)+ k{x - y) f {y) dy 

Jo 

with the kernel of the integral operator given by 

ZTT J-oo 

The symbol (j(^) is defined on the real line R and it is assumed that a(^) — 1 G L^(R). 
If we think of Ai, . . . , A^r as the eigenvalues of a random Hermitian matrix in the classical 
Gaussian Unitary Ensemble (GUE) the Fourier transform of the distribution function for 
the random variable given by 

N 

Y^f{K/V2N) 

i=l 
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can be shown to be equal (after an appropriate scaling limit) to the Predholm determinant 



det WR{a) 

with symbol 

An application of a Szego type theorem, which computes these determinants asymptotically, 
shows that the random variable is asymptotically normal. This holds if the function / satisfies 
the conditions required by the Szego theorem. In particular, the symbol should not have 
jump discontinuities. In the case where the function / is a characteristic function the symbol 
has at least two jumps and the classical results do not apply. However, generalizations of the 
theorem starting with [7] and mostly recently with [6] have been obtained so that one can 
now compute the asymptotics of these determinants for symbols with jumps and also other 
singularities of a certain type. For a history of this problem, the reader is referred to [10, 11]. 
It is interesting to note here that the asymptotics for the case where jumps occur have a 
different form that make the associated distributions not asymptotically normal. For more 
information about the connection between the random variables and Fredholm determinants 
we refer to [1]. 

In the case of other random matrix ensembles other operators arise. In particular, for 
the so-called Laguerre ensembles of positive random Hermitian matrices, the operators of 
interest are not Wiener-Hopf, but Bessel operators, where the Fourier transform is replaced 
by the Hankel transform. In this case a Szego type theorem was proved as well for symbols 
that are smooth. These results are contained in [4] . 

In two important cases, the Bessel operators reduce to operators which are a sum or 
difference of truncated Wicner-Hopf and Hankel operators. These operators also act on 
L^(0, i?) and according to the rule 

where k is the same as above. We will denote them by 

WR{a)±HR{a). 

Generahzations of Szego type theorems for symbols that are not smooth have been very 
difficult to obtain for this class of operators. One of the main difficulties is that while one 
can attempt to apply localization techniques which reduce the problem of singularities to a 
special family of symbols, the determinants for this family are not explicitly known. 

It was exactly the analogous problem which delayed the generalization of the Szego 
theorem in the Wiener-Hopf case to singular symbols. However, the difficulty was overcome 
in [6] by finding alternative Fredholm determinant representations for the determinants of 
the truncated Wiener-Hopf operators and then comparing them to corresponding formulas 
for determinants of finite Toeplitz matrices. These matrices are defined by 

Tniv) — {Vj-k)j,k=0,...,n-l- 
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Here </7fe stands for the A;-th Fourier coefficients of a function </? defined on the unit circle. 
For these the asymptotics were, conveniently, known. In [6] it was shown that for certain 
classes of singular symbols the Toeplitz and Wiener-Hopf determinants were (up to a simple 
constant factor) asymptotically equal when R ~ 2n. The alternative representations were 
obtained by using an identity of Borodin and Okounkov for Toeplitz determinants [8] and 
its Wiener-Hopf analogue [2]. These formulas were only vahd for smooth symbols and so it 
was necessary to introduce a parameter to regularize the symbols, apply the identity and 
then take a limit. This gave exact formulas for the determinants for the singular symbols. 
The final step was to prove that these were asymptotically equal. 

In this paper we follow a similar path to find analogous asymptotic formulas for the 
truncated Wiener-Hopf -|- Hankel operators 

WR{a) + HR{a) 

in a case where a is the characteristic function of an interval and so has two jump disconti- 
nuities. Fortunately, the asymptotics for the corresponding Toephtz -|- Hankel matrices 

Tni^p) + H^i^p) = {{V^j-k) + {¥^j+k+l))j,k=0,...,n-l 

are known for these symbols [5] and thus the comparison can be made. 
The symbol of interest is 

^(0 = 

and the corresponding kernel is 



/g-27ria _■^^f sin {x - v) ^ siu (x + y) \ 
V TT (x - y) 7r{x + y) J ' 



Theorem. With a as above we have, for \^a\ < 1/2, 

det {WR{a) + HR{a)) - e-^^'"^ R-^"" 2^"' G(l - 2a) G{1 + 2a), 

where G is the Barnes G-function. 

Here is an outline of the paper. In the next section we derive Fredholm determinant 
representations for the determinants of Tn{(fi) + Hn{(p) and WR{a)+HR{a) in case of smooth 
symbols. In Section III we introduce a parameter to regularize our symbols and apply the 
formulas. We then take quotients and let the parameter tend to its limit, obtaining in this 
way an exact formula for the quotient of the determinants with the singular symbols. In 
Section IV we show that (up to a simple factor) they are asymptotically equal. 

The stated results yields asymptotic information about the random variable 

N 

J2f{X,/V2N) 
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where / is the characteristic function of the interval (0, 1). This random variable counts the 
number of eigenvalues in the interval (0, \/2N). The techniques here can likely be modified 
to apply to other symbols as well so that more general discontinuous symbols can be also 
studied. 

II. Formuleis for truncated determinants 

If A is an invertible operator on Hilbert space of the form identity+trace class then for 
projections P and Q — I — P we have 

det PAP = (det A) ■ (det QA-^Q). (1) 

This can be used to derive a formula of Borodin and Okounkov [8] for the finite Toeplitz 
determinants det T„((p) and an anologous formula for the finite Wicncr-Hopf opertors Wji{a) 
[2]. For the former P is the projection P„ onto the subspacc of £^(Z+), Z+ = {0, 1, . . . , }, 
spanned by z'^ (A; = 0, . . . , n — 1), and for the latter P is the projection onto the subspace 
L^(0, i?) of L^(0,oo). A proof of the Borodin Okounkov formula for Toeplitz determinants 
based on formula (1) implicitly is in [6] and explicitly is in [9]. 

Here we shall obtain analogous formulas for the determinants of the operators T„((y9) + 
Hn{^p) and WR^a) + Hii{(j). We assume about if and a that they have sufficiently smooth 
logarithms (with logo" also belonging to L^(R)). Thus they have factorizations ^{z) = 
(p^{z) ^^{z) and cr(^) = o"+(^) cr^(^), with factors extending analytically in the usual way to 
be analytic and nonzero inside and outside the unit circle and above and below the real line, 
respectively. Moreover, we will restrict ourselves to the case where the symbols are even, 
i.e., Lp{z~^) = ^p{z) and cri—^) = cr{C,)i since that is all we shall need later. In this case we 
can assume without loss of generality that the factors of the factorizations are related by 
(p~{z) = ip~^{z~^) and cr~((^) = cr+(— ,^). Formulas can also be obtained for non-even symbols, 
but they are more complicated. 

In order to estabhsh these formulas, let us recall that the Wiener-Hopf and the Hankel 
operators acting on -L^(0, oo) are defined by 

POO 

W{a) : f{x) ^ g{x) = f{x) + k{x - y)f{y) dy, 

Jo 

roo 

H{a) : f{x) ^ g{x) = / k{x + y)f{y) dy, 

Jo 

where k{x) is given as before and a — 1 e L^(R). The Toeplitz and the Hankel operator 
acting on £^(Z+) are given in terms of their matrix representations by 

T{(p) = {(pj_k)j^kGZ+, H{(p) = {(pj+k+l)j,kGZ+, 

where (pk are the Fourier coefficients of the function (f defined on the unit circle. 

If (p and a satisfy the above mentioned assumption and are even, then the identities are 

det (r,(99) + H^i^j) = G[^r det [T{^-') {T{^) + H{^))] det (/ + H{^./^+) g„), (2) 
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det {WR{a)+HR{a)) = det [W{a-') {W{a)+H{a))] det {I + QRH{a^/a+) Qr). (3) 

The constants G[</7] and G[a] are well known constants that appear also in the Szego type 
theorems, 

G[ip\ = exp ^ log (^(e^^) dx) , 
G[<7]=exp(i-|_Jog<7(0de). 

To prove (2) we write 

dei{TM+HM) = det P„(r((/?) + i/((^))P„ 

= det PnT{^+)Pr,T-\^+){T{^) + H{^))T~\^-)Pr,T{^-)Pn 
= GM" det P^T-\^^){T{^) + H{^))T-\^-)P^. 

We apply (1) with P = P„ and A = T-i((/?+) (T((/?) + H{if ))T-\if-). Then det ^ equals 
the second factor on the right side of (2) since T~^{(p~) ((/?+) = T{(p~^), and it remains 
to identify A~^. Of course 

=T{^-){T{^) + H{^)r'T{^^). 
The pleasant fact is that if ip is even and invertible then 

{T{cp) + H{cp))-' = T{^-') + H{^-'). 

(See [3], Sec. 2. For non-even (f there is a more complicated formula.) Since T{ip~^) — 
T~^{ip~)T~^{ip~^) this gives 

A-'^I + T{^-)H{^-')Ti^-^). 
We now use twice the general identity 

(where ■^(z) = -0(2;"^)) to obtain 

T{^-) H{^-') r((^+) = T{^-) H{^-' ^) = H{^- ^) ^ i/((^7<^+) 
since we have assumed ip+ — (p~. This proves (2) and the proof of (3) is analogous. 

Suppose symbols ip{z) and cr({) are related by 

If we use the fact about the Laguerre polynomials Lj{x) that 

5 



we see that the j, k entry of the matrix for W{a) with respect to the orthonormal basis 
fj{x) — ^/2e~^ Lj{2x) for -L^(0, oo) is equal to 

1 /"OO /"OO roo 

e-^^^^-y^a{Ofk{y)fA^)dydxdi 

ZTT J-oo Jo Jo 

- h L ^<^«» We ^ h L ^^''> '''' i"'^!- 

(In the integral on the left z{^) denotes (1 + ~ ^0-) Thus the matrix for W{a) with 

respect to this basis is T{ip). (This fact is implicitly contained in [12]. Also, of course, 
the matrix for W{a^^) with respect to this basis is T{ip^^).) Similarly the j, k entry of the 
matrix for H{a) with respect to the same basis is equal to 

1 poo POD POO 

e~'^^-+y)a{Ofk{y)fj{^)dydxdC 
'^di 1 r , ,,,, ..N-,-fe-2 2(ie 



2ti J-oo Jo Jo 



POO •) fjf- POO 

= L oTW - L n 



= ^{z)z-^-'-^\dz\. 

ZTV J\z\=l 

Thus the matrix for H{a) with respect to this basis is H{ip). It follows that the first 
determinants on the right sides of (2) and (3) are equal and we have 

dei{WR{a) + HR{a)) ^ Gja]^ det {I + QRH{a^la+)QR) 

dct (T„(^) + iJ„(v9)) GM" dct (/ + Qn H{ip./ip+) g„) ■ ^ ^ 

In our application (4) will hold up to constant factors. This is enough for (5) to hold since 
those first determinants are unchanged if we multiply the symbols by constants. 

III. Fredholm determinant representations in the two-jump case 

Our Wiener-Hopf symbol is 

f 1 if lel > 1, 



This can be written alternatively as 



where the arguments of ^ ± 1 ± iO are zero when ^ > 0. The obvious way to regularize this 
is to replace by £ > everywhere and then eventually to take the £ — > limit. We use 
a different regularization which is more complicated at first but will be simpler in the end 
after we use the linear-fractional transformation 
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which maps the unit interval [—1, 1] to the upper half of the unit circle in the complex 
^-plane. To regularize we introduce a parameter r e (0, 1) and define 

where the arguments of the four factors are close to zero when ^ is large and positive. Since 

^(r) = l + i(l-r) + 0((l-r)2) as r ^ 1 (7) 

we see that o'r(^) — ^ (t{C,) as r ^ 1 when ^ ^ ±1. 
The Wiener- Hopf factors of ar{i) are 



SO that 



We shall think of QRH{a'' /a^) Qr as acting on 1/^(0, oo) rather than oo) by making 

the variable changes s — > it! + s, t ^ R + t. If we observe that (7^(0 — '^^ 
operator has kernel 



1 



■(0 



Wc deform the contour to the arc of the unit circle in the upper half-plane between ^(r) and 
— ^(r) = r) described in two directions (the "lower" part to the right and the "upper" 
part to the left). Then we are going to make the change of variable (6) so that the arc of 
the unit circle corresponds to the x-interval (— r, r). A computation shows that 



(e(x) + e(r)) {^x) + e(r)) _ (1 + rx) (r + x) 
(e(^)-e(r))(e(^)-e(r)) (l-rx)(r-x)' 

so that in particular the left side is purely negative on the arc. To compute its argument 
on the two sides of the arc, which will be independent of r by continuity, we can take r = 1 
where the geometry is simple, and find that the argument equals — tt on the upper part of 
the arc and, therefore, tt on the lower part. Thus 

(^ri^i^)) _^-i^a f {1 + rx) {r + x)Y 



aj. {^(x)) \{1 — rx) (r — x) ^ 

on the upper part of the arc and 

<(e(^)) ^ ..a / jl + rx) {r + x) \ 
[{l-rx){r-x)) 



7 



on the lower. Using these we find that (8) is equal to 

sinTra r / (1+r.x) (r + .x) r j^^^^2R+s+t) 2 
TT J-r\(l-rx)ir-x)J (1 + ixy 

The operator i^ij,^ with this kernel equals a product UV where U : L'^{—r, r) L'^{0, oo) 
has kernel 

^ ' ^ TT 1+ix 

and y : -L^(0, oo) — > L'^{—r, r) has kernel 

^ ^ / (r + x)(l + rx) r 2 ^,g(,H2fi+3) 

The determinant of I+Kji^r is unchanged if i^Tjij^^ is replaced by VU : L'^{—r, r) — > L'^{—r, r), 
which has kernel 

^sinTra ^2i€(a;)i? (1 + rx) (r + a:) ^" 2 i 



TT V(l -rx) (r-x)y (1 + (1 + ^(x) 

sin vra / (1 + (r + x) \ " e^*^*^^^^ 



TT V (1 ~ ''^'x) (j- — x) ) 1 + xy 
The determinants will also be unchanged if we change the kernel to 

' (1 + rx) (r + x) (1 + ry) {r + y)\ « 



. sm na 



TT \ (1 ~ ^2;) (r — x) (1 — ry) {r — y) ) 1 + xy 



(9) 



Next, before we take the r ^ 1 limit, we go to Toeplitz and the symbol 



if f < ^ < f . 



Thus (p{z) = e on the right half of the unit circle and (p{z) — e*'^" on the left half. This 
can also be written as 

, , ( \ — iz 1 — iz~^ \ 

where the arguments are chosen so that the factors involving z have argument at 2; = and 
those involving z~^ have argument at 2; = 00. To regularize we choose the same parameter 
r as before and replace ip{z) by 

/ 1 — irz l — irz~^\°' 
We point out here that if z{^) = (1 + ^0/(1 ~ ^0 then a computation shows that 

1 +ir^ 2a 
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Thus (4) holds for the symbols (7^(0 and (pr{z), up to constant factors, and therefore we 
have the identity (5). 

To continue, the Wiener-Hopf factors of (pr are 

— irz\ , , I — irz \ 



1 + irz 



-1 



so that 

^~{z) \{l-\-irz){l — irz-^)j 
If we think of Qn H[ip^ /ip^) as acting on £^(Z+) then it has j, k entry 

2'ni J (p~[z) 

with integration over the unit circle. (Wc use here the fact that ^ri^) — vti^'^)-) With 
the variable change z —>■ iz the above becomes 



2n J [(l-rz)(l-rz-^) 



XI -rz) (l-rz-^) 

The integrand is analytic inside the unit circle cut on the line segment [— r, r] . We deform 
the path of integration to this segment described back and forth. The expression in large 
parentheses is real and negative on the segment. The hmit of its argument from above equals 
— TT and from below equals tt. Hence 

HnAj,k) = r ( a + rx)(r + x) y 

TT J —r — TXj [T — Xj I 

This operator is now a product UV where U : L'^{—r,r) — > £^(Z+) has kernel 

TT 

and V : £^(Z+) L'^{—r,r) has kernel 

y(l — rx) [r — X) J 

For det(/ + i^n,r) this may be replaced by VU : L'^{—r, r) — >■ L'^{—r, r) which has kernel 

2^V[x,j)U[j,y) ^ — — . 

TT \{1 - rx) {r - X) J 1 + xy 



Without affecting the determinants this can be changed to 

.sin ttq; / (1 + rx) (r + x) (1 + ry) (r + y)\ {—xy)" 
TT \{^ ~ fx) (r — x) (1 — ry) {r — y) J 1 + xy 

9 



(10) 



We change notation and now denote by KR^r{x,y) and Hn^r{x,y) the kernels (9) and 
(10), respectively. They are independent of r but the corresponding operators both act on 
L^(— r, r) and we have shown that 

det jWRjar) + HRjar)) ^ Gfa,]^ det (7 + K^,,) 
det (TnM + HnM) G[iprY dct (7 + Hn,r) ■ 

Suppose we knew that 7 + Hn,r was invertible. Then we could rewrite the above as 

det(iy^(a.) + 77^(a.)) _ . ,^ 

det(r4^.) + 774^.)) - ^ ""^^ ^ ^ "'^^ 

Denote by TT^ and 77° the corresponding operators on L^(— 1, 1). We shall show in the next 
section that 7 + 77^ is uniformly invertible for large n and a trivial modification shows that 
7 + Hn^r is uniformly invertible if in addition r is bounded away from zero. We shall also 
show that K'^ — 77° is trace class. Granting these things in advance, the above holds and in 
addition we deduce the limiting relation 

We used here that G[ip\ = 1 and G[a] = e''^^'^. 

IV. Asymptotics 

Theorem 2.4 of [5] states for our Tn{ip) + Hn{(p) that 

det (T„((/?) + 77„((/?)) ~ n-=^"' 2^"' G{1 - 2a) G{1 + 2a) 

as n ^ oo. Thus our Theorem will be proved if we can show that (11) is valid and that 
the determinant on the right side there tends to 1 as n tends to infinity in such a way that 
n ~ 7?. These will follow once we show that — 77° tends to in trace norm and that the 
operators 7 + 77° are uniformly invertible.^ 

For the first, we observe that the "problem points" for our kernels are the pairs 
X — ±1, y — ^1. More precisely, for any S > the operator with kernel 

1 - [X(i-5, 1) jx) X(-i, -1+5) jy) + X(-i, -1+5) jx) X(i-s, 1) jy)] 

1 + xy 

is trace class, and the same is true even with the factor 

/ 1 + .T l+y V 
\l~x l~yj 

^In fact we shall show that the latter holds for all a satisfying |3?a| < 1/2 except for those lying in a 
discrete set. The extra condition on a can be removed at the end by an easy analyticity argument as at the 
end of [6]. We shall also assume that R — n = o(n^/^). This is good enough since any n = n{R) satisfying 
n{R) ~ i? as i? ^ oo would give us the asymptotics. 
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long as \^a\ < 1/2. Moreover multiplication by converges strongly to as n — > oo. It 
follows that 

(By this we mean that the left side is the kernel of an operator whose trace norm is o(l).) 
A similar argument applies to Kj^, so we are left with showing that the kernels 

X^,-s,i){^)X^-,,-,+s){y){KUx,y)-H^ni^,y)), X(_i,_i+5)(x)X(i_5,i)(y)(X°(x,y)-i/°(a;,y)) 

are oi(l). We consider only the first, and it is convenient to make the variable change y — > —y 
so that it becomes a constant times 

Lemma 1 of [6] says, with slightly different notation, that the trace norm of a kernel 
+ ^) -^^(0) oo) is at most a constant depending on h times the square root of 



1 1/(01^^ • / biv)?:^, 



^l+b J i-^v'/i ^l-b' 

Here b belongs to (—1,1) but is otherwise arbitrary. If we make the substitutions 

1-x 1-y 
1+x 1+y 

we find that the trace norm of a kernel F{x)G{y)/{l — xy) on L^(0, 1) is at most a constant 
times the square root of 

l\l-x)-'-'{l + xf-'\F{x)\^dx ■ [\l-yy-'{l + y)-'-'\G{y)\'dy. (13) 
Jo Jo 

If we write 

= (e^ - x") ^ + x" {e' ^^^-^^^'^ - y") , (14) 

then (12) is written correspondingly as the sum of two operators of the above form. In the 
first 

F(.)=X(,_,,)(x) (i±|)"(e'(«^M)^-x^), 

The second integral in (13) is 0(1) as long as 6 > —2^a. (Recall that ^ ^(—y) > 0.) 
We choose b = —2^a + e with £ > small enough so that the condition 6 < 1 is satisfied. 
We are left with showing that 



(1 - x)-^-' |e'«W-i)^ - x^'l^dx = o(l). 
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It is enough to show this with e replaced by 1/2, say. Using (7) and the variable change 
a; —> 1 — X we see that we want to show 



X 



-3/2 |g(-.+0(x2))iJ _ ^(_.+o(.2))n|2^^ ^ ^^^y 



The integral over x > n is exponentially small (if S was chosen small enough so that the 
O(x^) terms are less than x/2) and the rest is at most 



-2/3 



r x-^/^ \x (n-R) + 0(xW dx = 0(n-\n - Rf + n'^'^), 
Jo 

and this is o(l) if i? — n = o(n^/^). 

For the operator arising from the second summand in (14), in particular its second factor, 
we use the fact that C{~y) = ~C{y) then (7) again. We choose b = + 1/2 in (13) 
and the same integrals arise as before. This completes the proof that — = Oi(l). 

It remains to establish the uniform invertibility of / + for all a satisfying \^a\ < 1/2 
except for those lying in a discrete set. Recall that we only change the operator by oi(l) if 
we multiply the kernel by 

X(o, 1) (x) X(_i, 0) (y) + X(-i, 0) (x) X(o, 1) iv) 

so this will not affect uniform invertibility. Think of as already having this factor and 
then think of L^(— 1, 1) as L^(0, 1) © L^(— 1,0). The new will have a certain matrix 
representation. If we make the variable change x —x in L^(— 1,0) so that it becomes 
L^(0, 1), then all operators in the matrix act on L^(0, 1), and the matrix kernel for i/^ 
becomes 

\l-x 1+y J 1+xy 



.sm na 



TT 



I (l-xl±yY 
\ \l+x 1-yJ 1+xy 

First we make the same variable changes 

1 







/ 



X 



X 



1 + x' 

as before, and the matrix kernel becomes 

/ 



1-1/ 

1 + y 



. sm na 



TT 



(l-x 1-yY (x/y) 
\l+x 1+y J 



x+y 



(1-xi^Y 

\l+x 1+y) 





x+y 



J 



still acting on L^(0, 1) © iv^(0, 1). Then, using the lemma quoted from [6] and computations 
analogous to those already done, we can show that the error incurred is 0\{1) if we replace 

f l-x i_yy 

\l + x 1 + y J 
12 



by e gQ ^j^jg ^ot affect uniform invertibility. Then we make the substitutions 

X — > x/2n, y — > y/2n and the kernel becomes 



smTra 
( 

TT 



V 







x+y 



-{x+y) (x/y)" 
x+y 







on L2(0,2n) ©L2(0,2n). 

This is a variant of the operator that appears in Lemma 3.2 of [6]. The argument is 
essentially the same here, and we outline it. It suffices to show that / plus the limiting 
operator on L^(0, oo) © -£'^(0, oo) is invertible, except for a discrete set of a. Call the kernel 
L{x, y). First we show, by transforming to a Wiener-Hopf operator with matrix kernel, that 
/ plus the operator with kernel 



Loix,y) 



smira , , , . 



(xi 



x+y 



x+y 







is invertible except for a discrete set of a. Then we observe that L — Lq is a trace class 
operator. Putting these together we see that the invertibihty of / + L is equivalent to the 
invertibility of / + (/ + Lo)~^[L — Lq), which in turn is equivalent to the nonvanishing of 
its determinant. We know this does not vanish if a is sufficiently small and so it can only 
vanish for a discrete set of a. 
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